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PREFACE 


The  Institute  for  Defense  Analyses  (IDA)  prepared  this  document  for  the  Office  of 
the  Director,  Program  Analysis  and  Evaluation,  under  a  task  entitled  “Cost  Research 
Symposium.”  This  document  contains  an  assessment  of  DoD’s  capabilities  to  estimate  the 
costs  of  weapon  systems.  The  assessment  was  originally  presented  by  a  panel  of 
representatives  from  the  Office  of  the  Secretary  of  Defense  and  the  Military  Departments 
at  the  31*‘  Annual  DoD  Cost  Analysis  Symposium  conducted  on  February  3-6,  1998,  in 
Williamsburg,  Virginia. 

This  document  is  in  the  form  of  an  annotated  briefing,  augmented  with  additional 
information,  including  lists  of  offices  who  provided  data  the  panelists  used  to  develop 
scores  for  DoD’s  capabilities  by  commodity  type  (e.g.,  aircraft,  ships,  etc.). 

The  panelists  and  their  contributions  to  the  presentation  are  as  follows; 

•  Stephen  J.  Balut,  IDA — introduction  and  closing; 

•  Vance  Gordon,  CAIG — OSD  perspective; 

•  Theresa  O’Brien,  AFCAA — space  systems,  fixed-wing  aircraft,  and  rotary- 
wing  aircraft; 

•  Richard  Bishop,  USACEAC — missiles  and  surface  vehicle  systems;  and 

•  Richard  Collins,  NCAA — electronics  and  ships. 

Because  it  contains  no  original  analysis,  this  document  did  not  undergo  internal 
EDA  review. 
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I.  INTRODUCTION 

Stephen  J.  Balut,  Institute  for  Defense  Analyses 


ISatus  of  DqD%  CdpabilRy 

to 

Estimate  the  Costs 


of 


A  Panel  Discussion 


Good  morning.  Welcome  to  the  Cost  Research  Session. 

As  you  know,  the  purpose  of  cost  research  is  to  develop  and  improve  the 
data  and  methods  we  use  to  conduct  cost  analyses. 

Here’s  what  I’m  going  to  do  before  I  turn  the  floor  over  to  our  panel. 
First,  we’re  going  to  have  a  short  quiz.  Then  I  will  tell  a  short  story  that 
illustrates  why  we  do  cost  research.  Then  I’ll  describe  the  DoD  Cost  Research 
planning  cycle.  Finally,  I  will  introduce  the  panel  discussion  by  telling  you  what 
will  be  assessed,  the  question  being  answered,  the  dimensions  of  the  question, 
the  scoring  method,  what  our  prior  expectations  were,  and  lastly,  I  will 
introduce  each  member  of  the  panel. 

I  know  many  of  you  attended  our  earlier  sessions  on  cost  research.  I 
want  to  find  out  how  well  you  were  listening.  We’re  going  to  have  a  quiz — a 
short  one — one  question.  Let’s  see  who  gets  the  right  answer. 

Last  year,  I  told  you  about  my  trip  to  West  Africa  to  see  my  son  who 
was  serving  in  the  Peace  Corps.  He  taught  me,  and  I  told  you,  the  Bambara 
word  for  “estimate,”  along  with  its  literal  translation.  I  doubt  anyone  would 
remember  the  Bambara  word — its  “jatimine  (jaw-tee-mee-nay).”  But  someone 
might  remember  the  literal  translation.  The  quiz  question  is — what  is  the  literal 
translation  of  “jatimine?”  Anyone?  Shout  out  the  answer.  [Response  from  the 
audience — “Grab  a  number.”] 
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Unlike  the  story  last  year,  this  one  is  completely  fictitious.  I  use  it 
only  to  make  a  point.  Here’s  the  hypothetical  situation. 

Someone  walks  into  your  office  and  asks  you,  “How  much  will  the 
JSF  cost?” — and  the  answer  is  wanted  right  away.  The  answer  you  give  will 
depend  on  the  information  available  to  you  at  that  moment.  This  set  of 
information,  whatever  it  is,  might  include  data  on  the  JSF,  similar  data  on 
analogous  systems,  and  cost  estimating  methods  that  relate  the  tactical 
aircraft  program  characteristics  to  costs. 

If  at  the  moment,  you  have  little  or  no  information  related  to  the  JSF, 
you  could  “grab  a  number.”  That  is,  not  having  data  and  methods  will  not 
stop  you  from  offering  up  a  number.  You  might  say,  “About  $35  million  per 
copy,  but  I  have  no  hard  evidence  to  back  up  that  number.”  In  this  case,  the 
credibility  of  your  grabbed  number  estimate  is  about  zero.  It  will  be  of  little 
or  no  use  to  the  requestor. 

Let’s  suppose  a  different  situation.  Suppose  you  do  have  information. 
You  go  to  your  safe,  pull  out  data  on  the  JSF  (i.e.,  physical,  technical, 
performance,  programmatic,  and  cost)  and  also  on  analogous  systems.  Then 
you  go  to  your  bookshelf  and  select  the  preferred  method  for  estimating  the 
costs  of  tactical  aircraft.  You  develop  an  estimate  using  these  tools  and  also 
use  statistical  methods  to  determine  the  extent  of  error  in  your  estimate  (as 
determined  by  the  data  and  methods  you  used).  You  deliver  your  new 
estimate  to  the  requestor.  Let’s  say  it’s  $30  million  per  copy,  and  you  provide 
a  confidence  band  around  this  point  estimate  and  identify  the  sources  of 
uncertainty.  You  show  the  requestor  the  data  you  used  and  the  methods  you 
applied.  Now  your  estimate  is  useful  and  has  some  degree  of  credibility. 

In  these  scenarios,  your  capability  to  estimate  the  desired  costs  was 
determined,  in  large  part,  by  the  data  in  your  safe  and  the  methods  on  your 
bookshelf.  These  data  and  methods  were  the  results  of  prior  investments  in 
cost  research. 

This  story  illustrates  that  the  current  level  of  our  capabilities  to  do 
cost  analyses  and  estimate  the  costs  of  weapon  systems  is  no  accident.  They 
have  been  determined,  in  large  part,  by  prior  investments.  Likewise,  our 
future  capabilities  will  be  determined  by  the  investments  we  make  today  and 
tomorrow. 

Cost  research  dollars  are  very  scarce  and  must  be  carefully  planned. 
Our  investment  decisions  must  be  informed  in  several  ways.  We  need  an 
understanding  of  our  current  capabilities;  we  need  a  view  of  where  our  needs 
for  improvement  are  greatest  in  light  of  pending  challenges;  and,  just  as 
important,  these  decisions,  which  we  make  in  a  decentralized  way,  need  to 
be  informed  with  visibility  into  where  other  research  sponsors  are  making 
their  investments. 
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COST  RESEARCH  (CR)  PIJWNINO  CYCLE 

DoDCAS 

Start  new 

Assess  status  of  cost 

CR  projects 

~~~:^^^nalysis  capabilities 

w 

Revise  6^Year 

Lsep 

CR  Roadmap 

May/ 

- ^IDA/CAIG  CRS 

0^ 

Review  status  of  CR 

Allocate  CR  $ 
to  new  projects 

This  slide  shows  the  cost  research  planning  cycle  that  has  evolved  in  the 
DoD.  The  process  imposes  some  order  and  even  efficiency  on  the  process  by 
which  sponsors  choose  to  invest  their  scarce  cost  research  dollars.  The  two  main 
events  in  this  cycle  are  the  DoDCAS  and  the  IDA/CAIG  Cost  Research 
Symposium.  You  are  familiar  with  DoDCAS.  You’re  here.  At  this  meeting  we 
learn  the  status  of  DoD’s  cost  analysis  capabilities — through  meetings,  training 
sessions  and  panel  discussions. 

Some  of  you  may  not  be  familiar  with  the  IDA/CAIG  Cost  Research 
Symposium.  It  was  initiated  to  answer  the  question,  “What  cost  research  is 
going  on  today — and,  to  the  extent  it’s  known,  what  is  planned  for  tomorrow?” 
This  symposium  started  nine  years  ago.  I  was  sitting  at  my  desk,  thinking  about 
what  I  was  going  to  spend  my  independent  research  dollars  on.  I  realized  I  knew 
nothing  at  all  about  what  other  offices  (military  departments,  OSD,  other 
FFRDCs,  universities)  were  doing  now  or  what  they  were  planning  to  do.  I 
picked  up  my  phone  and  invited  my  colleagues  to  come  to  IDA  and  exchange 
information.  We  did  that.  Our  meeting  resulted  in  more  informed  decisions  on 
what  investments  to  make.  In  addition,  we  exchanged  data  and  findings  and 
even  decided  to  jointly  fund  certain  research  projects  of  common  interest. 
We’ve  been  meeting  this  way  ever  since.  The  CAIG  started  co-sponsoring  this 
meeting  in  1993. 
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So,  [referring  to  the  slide]  the  planning  cycle  starts  with  an  assessment 
of  cost  analysis  capabilities — here  at  DoDCAS.  This  results  in  identification  of 
areas  where  more  research  is  needed.  You’ll  see  these  areas  in  a  few  minutes. 
The  Six- Year  Cost  Research  Plan  is  updated  during  the  spring,  based  on  what 
we  know  as  of  the  time  of  DoDCAS.  Then,  all  ongoing  cost  research  activities 
are  reviewed  and  cataloged  at  the  IDA/CAIG  Cost  Research  Symposium.  At 
about  this  time,  sponsors  with  cost  research  money  are  ready  to  make  their 
investment  decisions  for  the  next  fiscal  year.  At  this  point,  they  know  the  status 
of  existing  capabilities,  to  include  areas  where  more  research  is  needed,  and 
they  have  visibility  into  ongoing  research.  The  allocations  of  funds  to  new 
research  projects  are  made  in  the  summer,  and  new  research  projects  start  up 
with  the  beginning  of  the  new  fiscal  year. 
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Our  panel  is  going  to  present  to  you  their  assessment  of  capabilities  as  of 
right  now.  This  assessment  reflects  the  data  we,  the  entire  defense  cost 
community,  have  in  our  safes  and  the  methods  we  have  on  our  bookshelves 
right  now. 

This  assessment  will  not  address  all  areas  of  cost  analysis.  We  decided 
we  simply  did  not  have  enough  time  to  do  all  of  that  in  an  hour.  Our  assessment 
will  be  limited  to  the  DoD’s  capability  to  estimate  the  costs  of  weapon  systems. 
Assessments  were  derived  by  first  talking  to  the  people  in  the  DoD  who  actually 
do  these  estimates — and  then  aggregating  their  individual  subjective  judgments. 

Now,  let’s  be  clear  on  what  is  not  included  in  today’s  assessments.  They 
do  not  explicitly  include  the  effects  of  the  so-called  revolution  in  business 
affairs,  the  effects  of  acquisition  reform,  of  acquisition  streamlining,  of  IPTs 
and  the  lik6.  These  effects  are  being  studied  now  and  have  yet  to  be 
incorporated  into  our  cost  estimating  toolbox. 

Also,  our  focus  on  weapon  systems  excludes  automated  information 
systems,  force  costing,  and  infrastructure  costing. 
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SCENARIO 

■  Situation: 

-  You  are  responsible  for  estinnating  the  cost 
of  a  weapon  system  in  preparation  for  a 
major  nrillestone  review. 

■  Question: 

--  How  well  are  you  prepared  to  do  this 
today? 


This  slide  shows  the  question  that  was  put  to  cost  analysts  in  DoD 
offices  that  are  responsible  for  estimating  the  costs  of  weapon  systems.  It  asks 
for  a  subjective  assessment  of  capability  to  estimate  the  costs  of  a  specific 
weapon  system  at  the  time  of  a  specific  milestone  decision.  For  example,  “How 
good  is  your  capability  to  estimate  the  cost  of  a  tactical  aircraft  at  the  time  of  an 
EMD  (milestone  E)  decision?”  Or,  “How  good  is  your  capability  later,  at  the 
time  of  the  production  milestone  decision?”  One  would  expect  capability  to  be 
better  at  the  production  milestone  because  more  data  would  be  available — to 
include  costs  experienced  during  EMD  and  LRDP. 
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DIMENSIONS 

■  Systems 

■  Milestones 

-  Space  Systems 

“  PDRR 

-  Aircraft 

-  EMD 

Electronics 

-  Production 

-  Ships 

-  O&S 

-  Missiles 

-  Surface  Vehicle 
Systems 

Assessments  will  be  provided  for  all  major  commodities  included  in 
Military  Standard  88 IB,  except  ordnance.  We  prepared  an  assessment  of 
ordnance,  but  the  number  of  people  who  submitted  their  judgments  was  small, 
and  the  results  raised  questions.  We  will  try  to  do  a  better  job  on  ordnance  next 
time. 

Assessments  will  be  provided  for  the  three  major  milestones  and  also 
for  O&S.  The  question  put  to  the  experts  about  O&S  costs  was  not  related  to 
any  specific  milestone,  but  rather  just  a  general  assessment. 
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SCARING 

■  GREEN-  Capabilities  good  or  better 

-  Adequate  data  avaitable 

-  CERs/Models  available  and  up-to-date 

-  Expect  small  to  moderate  error  In  estimates 

■  y'  JjJ.  Capabilities  marginal 

-  Some  data  available  -  additional  data  needed 

-  CERs/Models  available  but  not  current 

^  Expect  moderate  to  large  errors  in  estimates 

■  ftED  -  Capabilities  poor 

-  Data  lacking 

-  CERs/Models  not  available  or  of  little  use 

-  Expect  large  to  unknown  errors  In  estimates 


Here  is  the  color  coding  used  by  the  experts  to  score  their  assessments. 
Green  means  capabilities  are  believed  to  be  good  or  better.  This  means  adequate 
data  are  available  now,  CERs/models  are  available  now,  and  we  feel  that  the 
error  in  estimates  will  likely  be  small  to  moderate. 

Yellow  indicates  a  feeling  that  capabilities  are  marginal.  This  means  we 
don’t  have  all  the  data  we  need,  CERs  are  around  but  may  not  be  current  or 
directly  applicable,  and  we  might  have  moderate  to  large  errors  in  our  estimates. 

Red  says  our  capabilities  are  poor.  Data  are  lacking,  CERs/models  are 
of  little  use,  and  we  suspect  our  estimates  may  contain  errors  that  are  large  or 
worse. 

We  allowed  (and  our  responders  submitted)  assessments  that  included 
in-between  points.  So,  you  will  see  assessments  such  as  red-yellow  and  yellow- 
green.  These  mean  capabilities  are  judged  to  be  not  as  bad  as  the  left  (or  first) 
color,  but  not  as  good  as  the  right  (or  second)  color. 
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PDRR  Characteristics 

m  Utde  definition  of  program  being  estimated 

■  Paucity  of  historical  cost  data 

“  Limited  hardware/software  of  varying  configurations  (eg., 
breadboard,  brassboard,  etc,) 

-  Nonrecurring  vs  recurring  hardware  cost  visibility  often 
unavailable 

-  Noderate-to-significant  internal  contractor  cost  contribution 
not  reflected  in  the  data 

■  Limited  analogies  &  few-or^no  /77ea/7Ai7^/?//estlmating 

relationships 


The  next  four  slides  identify,  by  phase,  the  scores  we  expected  and  the 
reasons  why. 

At  Milestone  I,  the  decision  to  enter  the  PDRR  phase  (previously  known 
as  Demonstration  and  Validation),  we  expected  a  red-yellow  score  for  DoD’s 
PDRR  cost  estimating  capability. 

First,  the  program  being  estimated  tends  to  be  ill  defined  with  respect  to 
the  technical  baseline. 

Second,  historical  weapon  systems  databases  suffer  from  a  severe  lack 
of  PDRR  data.  Several  factors  contribute  to  this  void.  There  is  generally  a 
limited  amount  of  deliverable  hardware  and  software  developed  during  this 
phase,  and  often  the  hardware  built  is  not  a  full-up  system.  This  is  practically 
always  the  case  for  electronics,  whose  PDRR  phase  typically  involves 
development  of  breadboard  and  brassboard  hardware  for  potentially  risky 
subsystems  and  components.  Another  factor  that  contributes  to  the  PDRR  data 
void  is  the  quality  of  deliverable  (i.e.,  to  the  government)  cost  data.  The  most 
common  problem  is  lack  of  detail,  which  results  in  inability  to  distinguish 
between  nonrecurring  and  recurring  hardware  (i.e.,  design  vs.  build)  costs.  The 
final  factor  contributing  to  the  PDRR  data  void  is  that  contractor  costs  reported 
to  the  government  do  not  include  what  is  quite  often  moderate  to  significant 
internal  contractor  investment  in  the  PDRR  effort.  To  explain,  typical  program 
acquisition  strategies  include  PDRR  contract  awards  to  at  least  two  competing 
contractors  or  contractor  teams  and  subsequent  competition  for  a  single  EMD 
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award.  The  desire  to  get  the  competitive  edge  in  preparation  for  the  downselect 
is  a  strong  incentive  for  a  PDRR  contractor  to  expend  internal  funds. 
Understandable  though  unfortunate  for  cost  analysts,  there  is  no  requirement  for 
the  contractor  to  disclose  these  internally  funded  costs.  The  absence  of  these 
costs  from  our  historical  databases  has  a  negative  impact  on  our  ability  to  truly 
understand  PDRR  costs  and,  perhaps  more  importantly,  their  relationship  to 
EMD  phase  costs. 

Third,  as  a  result  of  the  quantity  and  quality  of  PDRR  data,  there  exist 
some  macro  cost  estimating  analogies  and  factors,  but  few  or  no  meaningful 
relationships  that  explain  PDRR  costs. 
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EMD  Characteristics 

■  Umited-to-moderate  definition  of  program  being 
estimated 

■  Wealth  of  historical  cost,  technical  8t  programmatic 
date 

-  Extensive  historical  EMD  data,  ijui  nonrecurring  vs.  recurring 
"noise"  hinders  comparability 

-  PDRR  data  for  program  being  estimated 

■  Analogies  &  est'mating  relationships 

-  Some  are  meaningful  &  promote  understanding 

-  Others  (e,g.,  factors  and  physical-based  CERs)  just  answer 
the  mail 


Capabilities  marginal 


At  Milestone  II,  the  decision  to  enter  the  EMD  phase,  we  expected  a 
yellow  score  for  DoD’s  EMD  cost  estimating  capability. 

First,  the  program  being  estimated  tends  to  be  better  defined  now  (vice 
at  Milestone  I)  with  respect  to  the  technical  baseline.  In  fact,  program  definition 
can  be  generally  characterized  as  limited  to  moderate. 

Second,  historical  weapon  systems  databases  include  a  wealth  of  EMD 
cost  data  and  associated  technical  (i.e.,  physical  and  performance)  and 
programmatic  data  from  which  EMD  cost  estimating  methodology  can  be 
developed.  Some  of  the  cost  data  (in  particular,  CCDR  data)  provide  visibility 
into  nonrecurring  vice  recurring  hardware  costs.  This  detail  enables  an  analyst 
to  distinguish  prototype  design  costs  from  prototype  build  costs,  a  distinction 
that  is  paramount  to  meaningful  EMD  cost  estimating  methodologies.  However, 
other  deliverable  cost  data  (i.e.,  CPR  and  CYSSR  data)  typically  do  not  provide 
similar  visibility.  In  addition  to  having  access  to  a  wealth  of  EMD  cost  data,  an 
analyst  preparing  an  estimate  in  support  of  Milestone  II  will  have  access  to 
PDRR  actuals  for  the  program  being  estimated.  The  previous  slide  highlighted 
the  limitations  associated  with  PDRR  data,  but  the  data  can  be  useful,  if  for  no 
other  reason,  as  a  sanity  check  of  specific  costs.  For  example,  the  systems 
engineering  costs  reported  for  PDRR  can  be  used  to  compute  an  actual  cost  per 
month  PDRR  bum  rate,  which  can  be  compared  to  the  estimated  EMD  bum  rate 
derived  from  historical  EMD  data. 


Ml 


Third,  due  largely  to  the  fact  that  significant  EMD  data  exists,  there  are 
a  variety  of  existing  and  potential  cost  estimating  methodologies  available  to 
cost  analysts  at  Milestone  H.  Some  of  these  methodologies,  which  include 
analogies,  factors,  and  statistical  parametric  relationships,  are  meaningful  with 
respect  to  explaining  costs.  Others  are  not  and  just  “answer  the  mail”  (i.e., 
produce  an  estimate). 
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Production  Characteiistics 

■  Good  definition  of  program  being  estimated 

!■  Wealth  of  historical  cost,  technical  8i  programmatic 
data 

-  Extensive  historical  production  data;  Iss  nonrecurring  vs 
recurring  "noise"  enables  better  comparability 

“  EMD  data  for  program  being  estimated 

■  Extrapolation,  analogies  8i  estimating  relationships 

-  Cost  vs.  learning  &  rate  relationships  well  established 

-  Better  relationships  between  HW  cost  &  performance 


At  Milestone  III,  the  decision  to  enter  the  Production  phase,  we 
expected  a  yellow- green  score  for  DoD’s  production  cost  estimating  capability. 

First,  the  program  being  estimated  tends  to  be  well  defined.  The  EMD 
Phase  is  completed  or  nearing  completion  and,  as  such,  the  technical  baseline  is 
maturing. 

Second,  historical  weapon  systems  databases  include  a  wealth  of 
production  cost  data  and  associated  technical  (i.e.,  physical  and  performance) 
and  programmatic  data  from  which  production  cost  estimating  methodologies 
can  be  developed.  Since  production  effort  is  primarily  recurring  in  nature,  the 
EMD  estimating  challenge  posed  by  the  nonrecurring  versus  recurring 
distinction  is  little  to  no  problem  for  production  estimating.  (Note:  an  exception 
is  special  tooling  and  test  equipment  cost,  which  does  have  an  important 
nonrecurring  component.)  This  provides  analysts  some  degree  of  confidence 
that  the  data  for  different  programs  are  comparably  defined.  Finally,  in  addition 
to  having  access  to  a  wealth  of  production  cost  data,  an  analyst  preparing  an 
estimate  in  support  of  Milestone  III  will  have  access  to  EMD  actuals  for  the 
program  being  estimated.  Depending  on  its  granularity,  this  EMD  data  can  be 
an  invaluable  resource  for  estimating  first  unit  recurring  production  hardware 
costs. 
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Third,  due  largely  to  the  fact  that  significant  EMD  and  production  data 
exists,  there  are  a  variety  of  existing  and  potential  cost  estimating 
methodologies  available  to  cost  analysts  at  Milestone  III.  Furthermore,  the 
reduced  (i.e.,  relative  to  EMD)  uncertainty  in  production  cost  data  alluded  to 
above  translates  into  reduced  uncertainty  in  the  cost  estimating  methodology 
derived  from  the  data  and  therefore  more  meaningful  methodology. 
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Characteristics 

Wealth  of  historical  cost  &  programmatic  data  via 

Services*  VAMOSC  systems/  but 

-  some  Important  cost  elements  are  missing  or  Incomplete 

-  level  of  cost  data  detail  not  always  adequate 

Analogies  &  estimating  Felatlonships  that  answer  the 

mail,  but  do  notfecllitate  understanding 

-  Relationships  between  cost  &  reliability,  maintainability  & 
availability  parameters  are  understood,  but  not  quantified 

-  Relationships  between  cost  and  system  technical  parameters 
are  not  understood,  no  less  quantified 

,  -  Limited  understanding  of  O&S  activities,  and  their  inter¬ 
relationships  and  cost  characteristics 


Capabilities  marginal 


At  Milestone  III,  the  decision  to  enter  the  Production  phase,  we 
expected  a  yellow  score  for  DoD’s  O&S  cost  estimating  capability. 

First,  the  Service’s  respective  VAMOSC  databases  include  a  wealth  of 
historical  cost  data  for  active  and  retired  weapon  systems.  These  databases  have 
formed  the  principal  basis  of  Service  and  OSD  CAIG  O&S  cost  estimates  for 
weapon  systems.  Each  database  has  a  well-defined  cost  element  structure  that 
facilitates  consistency  and  comparability  across  weapon  systems.  However,  the 
VAMOSC  databases  are  not  without  problems.  Some  O&S  cost  elements  (e.g., 
software  maintenance  and  engineering/technical  services)  are  missing  or 
incomplete.  Also,  in  some  instances,  the  data  do  not  provide  the  visibility 
required  to  perform  a  specific  estimate,  e.g.,  at  the  subsystem  or  component 
level. 

Second,  despite  the  wealth  of  historical  data,  there  is  a  paucity  of  O&S 
estimating  methodology  that  supports  anything  beyond  relatively  macro  cost 
estimates  that  just  “answer  the  mail”  (i.e.,  meet  Service  and  OSD  requirements 
to  estimate  life  cycle  cost).  Methodologies  that  represent  the  historical 
relationship  between  the  cost  (e.g.,  unscheduled  maintenance)  of  a  given  system 
and  the  system’s  demonstrated  reliability,  maintainability  and  availability 
(RMA)  parameters  are  lacking.  Similarly,  methodology  that  represent  the 
historical  relationship  between  a  given  O&S  cost  element  and  a  system’s 
performance  characteristics  (speed,  range,  etc.)  are  lacking.  Both  are  essential 
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to  DoD’s  ability  to  perfomi  analyses  that  support  the  CAIV  process  for  both 
future  and  in-service  weapon  systems.  Without  these  types  of  methodologies,  it 
is  difficult  (if  not  impossible)  to  conduct  the  cost-performance  trade-offs 
inherent  in  the  CAIV  process.  Finally,  there  is  a  basic  lack  of  understanding  of 
O&S  processes,  their  interrelationships  and  their  cost  characteristics.  This  final 
void  is  beginning  to  be  addressed  in  cost  modeling  efforts  that  employ  system 
dynamics. 
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PANEL 

m  Ms.  Theresa  O'Brien,  AFCAA 

-  Space  Systems 

-  Aircraft 

■  Mr.  RIckColitns,  NCCA 

Electronics 

-  Ships 

■  Mr.  Richard  Bishop,  CEAC 

-  Missiles 

-  Surface  Vehicle  Systems 

■  Dr.  Vance  Gordon,  OSD  CAI6 

-  Consolidation/CAIG  Perspective 

-  Upcoming  DAB  Schedule 


With  that  I’d  like  to  introduce  our  panel  and  get  on  with  the  assessment. 

Our  first  panel  member  is  Ms.  Theresa  O’Brien.  Theresa  is  the  Research 
Division  Chief  at  the  Air  Force  Cost  Analysis  Agency.  She  is  responsible  for  all 
the  Agency’s  cost  research  activities  and  cost  support  contracts.  Prior  to  joining 
the  Agency,  she  worked  at  the  Naval  Air  Systems  Command  on  the  F/A-18 
program.  Theresa  earned  a  master’s  degree  in  business  administration  from 
Virginia  Tech.  She  will  present  assessments  for  aircraft,  both  fixed  wing  and 
rotary  wing,  and  also  space  systems. 

Our  next  panelist  is  Mr.  Rick  Collins.  Rick  is  the  Technical  Director  of 
NCCA.  He  coordinates  Navy  cost  research.  Prior  to  his  role  as  Technical 
Director,  he  was  head  of  the  Ships  and  Ship  Systems  Division  of  NCCA.  Prior 
to  joining  NCCA,  Rick  worked  as  a  cost  analyst  at  SAIC.  Rick  has  a  master’s 
degree  in  economics  from  Virginia  Tech  and  a  bachelor’s  degree  in  economics 
from  Wake  Forest.  He  will  provide  assessments  for  ships  and  electronics. 

Our  next  panel  member  is  Mr.  Dick  Bishop.  Dick  is  the  Chief  of  Cost 
Research  at  the  Army  Cost  and  Economic  Analysis  Center.  He  analyzes  Army¬ 
wide  cost  research  requirements  and  develops  and  manages  the  Army’s  long- 
range  cost  research  program.  Dick  began  his  government  career  as  an  Army 
Signal  Corps  Officer.  He  later  worked  for  IBM  as  a  computer  designer.  Dick 
holds  a  BS  degree  in  electronics  engineering  and  an  MS  in  industrial 
engineering,  both  from  Oklahoma  State  University.  Dick  will  present 
assessments  for  missiles  and  surface  vehicle  systems. 
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Our  last  panelist  is  Dr.  Vance  Gordon.  Vance  is  a  member  of  the 
Operations  Analysis  and  Procurement  Planning  Division  of  PA&E’s  Resource 
Analysis  Directorate.  Since  joining  this  office,  Vance  has  been  responsible  for 
development  of  DoD  cost  research  guidance.  He  served  previously  in  PA&E’s 
Projection  Forces  Division.  Dr.  Gordon  is  a  graduate  of  the  University  of 
Colorado  and  received  his  Ph.D.  in  population  biology  from  Washington 
University  in  St.  Louis.  He  will  provide  a  consolidated  perspective  on  DoD’s 
capabilities  and  identify  some  future  challenges. 
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n.  SPACE  SYSTEMS 

Theresa  O’Brien,  Air  Force  Cost  Analysis  Agency 


Space  Systems 

PDRR 

EMD  Production  O&S 

Inteeration  Assembly  and  Test  miim  j - 1 - 1 

Software  Hi  1  Hi  1  11 

^H1 

Spacecraft  L_Hi  ^  HH 

1 . .u 

1  Hii  ^^^H  ^^^H 

r-rn 

Ground  C3  FH  1 

r:n 

1 

i:  r-:i 

Test  and  Evaluation  CZHB 

u _ 1 

rn— 1 

SE/PM  CZM  1 

ZIMM 

hhh 

Data  1 1 - 

JH 

1  1  J 

Training  ^^^H  I^^^H  HHHm  I  I  J 

Support  Equipment 

HIH 

1  rn 

Spares  1  1 

III 

1 

1  1  J 

Launch  Operations  and  Orbital  Spt  ^^^H 

■■ 

■■ 

1  1  1 

Launch  Vehicle  1  J 

LJ_J 

1  11 

The  efforts  my  colleagues  and  I  have  undertaken  bring  a  new  twist  to 
the  way  we  view  our  research  capability. 

First,  it  is  important  to  present  our  current  capabilities  in  space.  As  you 
can  see,  the  WBS  is  down  the  left  and  the  acquisition  phase  is  across  the  top. 
We  are  viewing  our  capability  at  each  milestone.  In  the  time  allocated,  we  can 
not  discuss  every  block,  but  only  highlight  the  findings.  To  support  the  color 
scheme,  we  have  also  documented  a  list  of  studies  that  are  commonly  used. 
This  list  of  studies  is  not  to  be  considered  inclusive  of  all  tools.  It  only 
represents,  in  no  particular  order,  the  studies  provided  by  the  respondents. 

Space  has  several  large  databases  that  comprise  historical  data. 
Unmanned  Spacecraft  Cost  Model  (USCM)  and  the  NASA/Air  Force  Cost 
Model  (NAFCOM).  These  databases  provide  analysts  with  valuable 
information;  thus,  the  green  coding  for  items  such  as  Integration  Assembly 
Test,  Spacecraft,  Payload,  Training,  and  Support  Equipment. 

Software,  as  you  will  see,  is  mostly  red  for  all  commodities.  Rick 
Collins  will  address  software  in  more  detail  in  the  Electronics  portion. 

Spacecraft  as  noted  above  relies  on  the  historical  databases.  But  trends 
such  as  the  common  bus  becoming  commercialized  causes  estimating 
challenges. 
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Payload  has  several  supporting  studies  for  use  in  cost  estimating.  The  Air 
Force  is  comfortable  estimating  sensor,  communication,  and  navigation 
payloads,  but  when  faced  with  new  technology  such  as  cross-links  payloads,  the 
Air  Force  relies  on  engineering  adjustment  factors. 

Ground  C3  systems  are  causing  estimating  challenges  in  the  area  of 
COTS  Integration. 

SE/PM  and  Data  are  coded  similarly  based  on  the  databases  of  historical 
data  not  capturing  acquisition  streamlining  initiatives  the  contractors  are 
implementing. 

Spares  estimates  are  typically  not  estimated  by  the  weapon  system 
analysts.  It  is  considered  a  pass-through  estimate  from  other  organizations.  No 
clear  methodologies  or  studies  were  provided. 

The  Launch  Vehicle  Cost  Model  is  a  valuable  resource  for  estimating 
EELV.  This  procurement  method  for  the  future,  using  price  negotiation  for 
launch  services  on  the  EELV  until  the  year  2020,  does  not  provide  insight  into 
the  cost  of  a  launch  vehicle.  The  government  is  acting  as  a  corporation  and 
purchasing  the  service  only.  If  a  future  decision  were  to  purchase  government 
owned  launchers,  there  will  be  no  historical  data  to  base  our  estimates. 

Some  areas  of  O&S  are  clear.  This  is  because  of  the  unique  nature  of 
space  systems.  When  a  satellite  is  launched  the  traditional  O&S  for  the 
spacecraft  are  not  relevant.  Maintenance  is  done  through  software  uploads  by 
the  ground  station. 
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Integration  A&T 


USCM,  Unmanned  Spacecraft  Cost  Model,  Tecolote,  1997,  (N  /  R) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997,  (N  /  R) 

NAVSTAR  GPS  Data,  SMC^C,  (N  /  R,  1  program) 

GPALs  CERs,TASC-Arlington,  Jan  93,  (N/  R  multi-programs) 

SEER  H,  Systems  Evaluation  &  Estimation  Resources-HW,  Galorath  Associates,  1997 
Space  Payload  Integration  Model,  Tecolote,  1994 

Software 

PRICE  S,  Martin  Marietta,  1997 

SEER  SEM,  Systems  Eval  &  Est  Resources-S/W  est  model,  Galorath,  1997 

CERs  for  Space-Based  Sys,  Defense  Communications  Agency-DC,  Apr  91,  (N  /R,  comm,  sys) 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90(SAV  productivity) 

Saset  Software  Architecture  Sizing  &  Estimating  Tool,  Martin-Marietta,  Apr  93 

Revic,  REVised  Intermediate  COCOMO,  AFCAA,  Feb  91 

SMC  Software  Sizing  Dbase,  SMC,  1997 


Spacecraft 

USCM,  Unmanned  Spacecraft  Cost  Model,  Tecolote,  1997,  (N  /  R) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997  (N  /  R) 

Revised  Small  Satellites,  Tecolote,  Nov  91,  (N  /Tl) 

CERs  for  Space-Based  Sys,  Defense  Communications  Agency-DC,  Apr  91,  (N  /  R,  comm.,  sys) 
S/C  Functional  CER,  IDA  for  BMDO,  Sep  93,  (N  /  R) 

NAVSTAR  GPS  Data,  SMC/FMC,  (N  /  R,  1  program) 

EPS  ECR,  Electrical  Power  Subsystem,  Booz  Allen,  Jun  91,  (N  /  Tl) 

GPALs  CERs,  TASC-Arlington,  Jan  93,  (N  /  R  multi-programs) 

High  Reliability  Parts,  MCR,  Sept  90,  (N/R/O&S) 

Elect  Power  Sys  for  SDIO  Elements,  Booz  Allen,  Jun  91,  (Streamlining) 

Large  Space  Power  Systems,  Aerospace  Coip,  Aug  88,  (N  /  R,  EPS) 

CERs  for  Prop  &  Reaction  Cntrl,  Applied  Research,  Feb  90,  (R) 

SEER  H,  Systems  Evaluation  &  Estimation  Resources-HW,  Galorath  Associates,  1997 
PRICE  H,  GE,  1997 

Small  Satellite  Subsys  Cost  Model,  Aerospace,  96,  (N/R) 

JPL  Project  Cost  Model,  Jet  Propulsion  L£6,  (N/R) 

TRANSCOST,  TransCost  Systems,  95  (N/R) 

Digital  Sig  &  Data  Processor,  DSDPM,  Tecolote  Research,  Sept  91,  (N/R) 

Phase  I  Acquisition  Reform,  TASC 1996 
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Payload 


USCM ,  Tecolote,  1997  (N  /  R) 

Price  H  /  M,  Martin  Marietta,  1997 
SEER  H.  Galorath,  1997 

Passive  Space  Sensor  Model,  MCR,  May  92,  (N  /  R) 

CERs  for  Space-Based  Sys,  Defense  Communications  Agency-DC,  Apr  91,  (N  /  R,  comm,  sys) 
Strategic  and  Exp  IR  Sensors,  Technomics,  Mar  93,  (R) 

Focal  Plane  Array  Cost  Est  Model,  Tecolote,  Jul  90,  (N  /  R) 

CER  for  R&D  Missile  Comm,  Applied  Research,  Mar  90 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90  (S/W  productivity) 
Scientific  Inst  Cost  Model-SICM,  Planning  Research,  91  (N/R) 

Digital  Sig  &  Data  Processor,  DSDPM,  Tecolote,  Sept  91,  (N/R) 

Non-Rec  paits(costs)  for  Space  Sensors,  Aerospace  for  SMC,  Oct  91,  (N) 

Adv  Space  Processor  Model,  Tecolote,  Sept  89,  (N/R) 

Dev  Eng  &  BTL  Dev  Models,  Technomics,  Aug  90,  (N) 

Tactical  IR  Sensor  Model,  Technomics,  Feb  91,  (R  small  payloads) 

CER  Rationale  for  BrillantEyes,  Technomics,  Apr  91,  (N/R) 

High  Reliability  Parts,  MCR,  Sept  90,  (N/R/O&S) 

Multi-var  Instrument  Cost  Model,  MICM,  90  (N) 

Ground  C3 

Construction  Cost  Est  Handbook,  Applied  Research,  Jun  88,  (N/R) 

GPALs  CERs,  TASC-Arlington,  Jan  93,  (N  /  R  multi -programs) 

Fiber  Optics  Network  Study,  General  Research  Corp,  Oct  89 
JPL  Project  Cost  Model,  Jet  Propulsion  Lab,  (N/R) 

Ground  Ops  Cost  Model-GOCM,  SAIC,  96  ^/R) 

Space  Ops  Cost  Model-SOCM,  SAIC,  (N/R) 

TRANSCOST,  TransCost  Systems,  95  (N/R) 


Test  and  Evaluation 

USCM ,  Tecolote,  1997  (N  /  R) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997,  (N  /  R) 

NAVSTAR  GPS  Data,  SMC/FMC,  (N  /  R,  1  program) 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90,  (S/W  productivity) 
GPALs  CERs,  TASC-Arlington,  Jan  93,  (N  /  R  multi-programs) 

Space  &  Strat  Def  Updated  CERs,  MCR,  Dec  87,  (N/R,  similar  to  Passive  Space  Sensor  Model) 
CER  rationale  for  Brilliant  Eyes,  Technomics,  Apr  91,  (N/R,  summary  of  other  methods) 

Dev  Eng  &  BTL  Dev  Models,  Technomics,  Aug  90,  (N) 

SE/PM 

USCM,  Unmanned  Spacecraft  Cost  Model,  Tecolote  1997,  (N  /  R) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997  (N  /  R) 

NAVSTAR  GPS  Data,  SMC/FMC,  not  known,  (N  /  R,  1  program) 

Focal  Plane  Array  Cost  Est  Model,  Tecolote,  Jul  90,  (N  /  R) 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90,  (S/W  productivity) 
GPALs  CERs,  TASC-Arlington,  Jan  93,  (N  /  R  multi -programs) 

Tactical  IR  Sensor  Model,  Technomics,  Feb  91,  (R  small  payloads) 

CER  Rationale  for  Brillant  Eyes,  Technomics,  Apr  91,  (N/R,  summary  of  other  methods) 

Data 

USCM,  Unmanned  Spacecraft  Cost  Model,  Tecolote  1997,  (N/R) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997  (N  /  R) 

NAVSTAR  GPS  Data,  SMC/FMC,  not  known,  (N  /  R,  1  program) 

Focal  Plane  Array  Cost  Est  Model,  Tecolote,  Jul  90,  (N/R) 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90,  (S/W  productivity) 
GPALs  CERs,  TASC-Arlington,  Jan  93,  (N  /  R  multi -programs) 

CER  Rationale  for  Brilliant  Eyes,  Technomics,  Apr  91,  (N/R,  summary  of  other  methods) 

Dev  Eng  and  BTL  Dev  Models,  Technomics,  Aug  90,  (N/R) 

n-4 


Training 


Support  Equipment 


USCM,  Unmanned  Spacecraft  Cost  Model,  Tecolote  1997,  (N  /  R) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997  (N  /  R) 

NAVSTAR  GPS  Data,  SMC/FMC,  not  known,  (N  /  R,  1  program) 

Focal  Plane  Array  Cost  Est  Model,  Tecolote,  Jul  90,  (N  /  R) 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90,  (S/W  productivity) 
GPALs  CERs,  TASC -Arlington,  Jan  93 ,  (N  / R  multi-programs) 

CER  rationale  for  Brilliant  Eyes,  Technomics,  Apr  91,  (N^,  summary  of  other  methods) 

Dev  Eng  &  BTL  Dev  Models,  Technomics,  Aug  90,  (N) 

USCM,  Unmanned  Spacecraft  Cost  Model,  Tecolote  1997,  (N  /  R) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997  (N  /  R) 

Mce  H,  Martin  Marietta,  1997 

Seer  H,  Systems  Evaluation  &  Estimation  Resources-H/W,  Galorath,  1997 
NAVSTAR  GPS  Data,  SMC/FMC,  not  known,  (N  /  R,  1  program) 

Focal  Plane  Array  Cost  Est  Model,  Tecolote,  Jul  90,  (N  /  R) 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90,  (S/W  productivity) 
GPALs  CERs,  TASC-Arlington,  Jan  93,  (N  /  R  multi-programs) 

CER  rationale  for  Brilliant  Eyes,  Technomics,  Apr  91,  (N/R,  summary  of  other  methods) 

Dev  Eng  &  BTL  Dev  Models,  Technomics,  Aug  90,  (N) 

Space  and  Strat  Def  Updated  CER,  MCR,  Dec  87,  (N/R,  similar  to  Passive  Space  Sensor  Model) 


-s 

Spares 

Kantors  Factors,  Cost  Factors  and  Est  Relationships,  Electronic  Sys,  Apr  90,  (S/W  productivity) 
GPALs  CERs,  TASC-Arlington,  Jan  93,  (N  /  R  multi-programs) 

Launch  Operations  &  Orbital  Spt 

USCM,  Unmanned  Spacecraft  Cost  Model,  Tecolote,  1997,  (N/R) 

Construction  Cost  Est  Handbook,  Applied  Research,  Jun  88,  (N  /  R) 

Space  and  Strat  Def  Upd£U:ed  CER,  MCR,  Dec  87  (N/R,  similar  to  Passive  Space  Sensor  Model) 
TRANSCOST,  TransCost  Systems,  95  (N/R) 

Launch  Vehicle 

Launch  Vehcile  Cost  Model,  Tecolote,  95,  (N/Tl) 

NAFCOM,  NASA/AF  Cost  Model,  SAIC,  1997  (N  /  R) 

Liquid  Rocket  Engine  Cost  Model,  Rockwell,  96,  (N/R) 

TRANSCOST,  TransCost  Systems,  95  (N/R) 

Digital  Sig  &  Data  Processor,  DSDPM,  Tecolote  Research,  Sept  91,  (N/R) 

— 
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III.  FIXED-WING  AIRCRAFT 
Theresa  O’Brien,  Air  Force  Cost  Analysis  Agency 


Aircraft  -  Fixed  Wing 

PDRR  EMD  Production  O&S 

Propulsion  [ZZ^H 

Zm  ■■■  [ZH 

Subsystems  (hydraulics,  flight  controls...)  1_J _ 1 

Tn  m  \^~rT 

Avionics 

m  1  I  I  J  L.  J:  ..:I 

Integration  Assembly  and  Test  |  ]  1 

Software 

m—]  r~i  1  tmm 

Armament 

■  1  1.1  I  III 

Test  and  Evaluation 

■  1  l,.l,  1  U_l 

SB/PM  1  H  1  H  1  ■  III 

Ejm  nim  njm 

Training 
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Fixed-Wing  Aircraft  is  the  next  commodity  to  be  discussed. 

An  abundance  of  historical  airframe  databases  is  available  to  the  cost 
analyst.  The  challenges  for  an  estimator  with  new  systems  revolve  around  the 
material  complexity,  stealth,  and  raw  material/purchased  parts.  Material  type, 
such  as  estimating  composites,  is  a  current  difficulty  on  systems  such  as  the 
JSF.  This  is  primarily  due  to  large  composite  systems  not  yet  under 
production.  Cost-estimating  relationships  are  currently  being  derived  to  assist 
in  this  area.  The  improvement  in  production  to  a  yellow/green  is  because  one 
can  rely  on  TRIP  units. 

The  propulsion  area  has  a  current,  detailed  database  available  to 
analyst.  The  challenge  is  capturing  the  cost  to  enhance  existing  systems. 

Subsystems  are  assigned  yellow  due  to  current  efforts  to  do  more  with 
less  space  and  using  integrated  systems  versus  federated  systems. 

For  Avionics,  new  integrated  systems  must  rely  only  on  F-22  EMD 
actuals.  The  studies  listed  on  the  following  slides  are  out-dated.  This  is  an 
important  area  for  focusing  future  research  efforts. 

Software  estimating  is  troublesome.  Tools  to  estimate  software  are 
available,  but  the  inputs  into  these  tools  have  not  been  collected  for  historical 
programs. 

For  armament,  analogies  are  used  but  there  is  no  study  or  database 
containing  these  analogies. 
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SE/PM,  Data,  Training,  and  Support  Equipment  are  coded  similarly 
based  on  the  databases  containing  the  historical  data  not  capturing  acquisition 
streamlining  initiatives  the  contractors  are  implementing. 

Aircraft  modification  challenges  are  embedded  in  the  coloring  scheme. 
The  challenges  in  structural  and  avionics  modifications  present  areas  requiring 
further  research. 

In  O&S,  one  survey  noted  that  part  obsolescence  has  become  a  difficult 
issue  for  the  estimator. 

Survey  respondents  for  fixed-wing  aircraft  include  AFCAA,  ASC, 
NCCA,  and  NAVAIR. 
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Litegration  Assembly  &  Test 

MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 

Cost  Factors  for  A/C  and  Missiles,  Aeronautical  Systems  Division,  May  87 

C3  Platform  Integration  Cost  Model,  MCR,  1997 

Kanter’s  Factors,  Cost  Factors  and  Estimating  Relationships,  Electronic  Sys  Division,  Apr  90 
Aircraft  Avionics  &  Missile  System  Installation  Cost  Study,  MCR,  1988 

A  Parametric  A/C  Avionics  and  Missile  System  Installation  Cost  Model,  MCR,  1986 

Price  H 

Standard  Cost  Factors  Handbook,  NCCA,  1992 

Airframe 

MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 

Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 

Composites/Exotic  Materials  Database,  Tecolote,  1997  (N/R) 

Advanced  Airframe  Structural  Materials,  RAND  Study,  1991 

Aircraft  Airframe  CEEis,  RAND,  87  (Total  Level) 

Military  Tactical  Aircraft  Development  Costs,  IDA,  1988 

Propulsion 

MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 

NAVAIR/AFCAA  Engine  Study,  Ketron,  1997  (N/R) 

Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 

GFE,  NAVAIR  Database,  1997 

Development  and  Prod.  Cost  for  Military  Aircraft  Turbine  Engines,  IDA,  1992 

Military  Tactical  Aircraft  Development  Costs,  IDA,  1988 

Subsystems 

MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 

Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 

GFE,  NAVAIR  Database,  1997 

Military  Tactical  Aircraft  Development  Costs,  IDA,  1988 

PRICE  H 

SEERH 

Avionics 

MACDAR  Hghter  Aircraft  Database,  Tecolote,  1997 

A  Data  Base  of  Airborne  Avionics,  Tecolote,  Jan  95 

Electronic  Box/EIectro-optical  Equip  Cost  Analysis  Brief,  Tecolote,  Sept  86 

Airborne  &  Ground  Mobile  Electronic  Box  Analysis,  Tecolote,  Sept  86 

Black  Box  Estimator  -  Electronics  Cost  Models,  Tecolote,  Nov  87 

Electronic  Systems  RDT&E  Cost  Model,  MCR,  May  88 

Cost  Impacts  of  Electronic  Boxes  due  to  Basing  Modes,  Tecolote,  Sept  87 

Radar  Production  Cost  Model,  MCR,  May  88 

Kanter's  Factors,  Cost  Factors  and  Estimating  Relationships,  Electronic  Sys  Division,  Apr  90 

Electronic  Subsystem  Integration  Estimator,  TASC,  Jul  85 

Military  Tactical  Aircraft  Development  Costs,  IDA,  1988 

Aircraft  Avionics  &  Missile  System  Installation  Cost  Study,  MCR,  1988 

GFE,  NAVAIR  Database,  1997 

Price  H,  HL,  M,  General  Electric,  1997 

SEERH 

Software 

Software  Development  Estimating  Handbook  -  Phase  One,  NCCA,  1998 

Price  S,  Parametric  Review  of  Info  for  Costing  and  Evaluation  Software  Sizing  Model,  GE,  1997 

SEER  SEM,  Systems  Evaluation  and  Estimation  Resources-SAV  Est  Model,  Galorath,  1997 

Kanter*s  Factors,  Cost  Factors  and  Estimating  Relationships,  Electronic  Sys  Division,  Apr  90 

Revic,  Software  Cost  Estimating  Model,  AFCAA,  Feb  91 

SASET,  Software  Architecture  Sizing  and  Estimating  Tool,  Martin  Marietta,  Apr  93 

Armament 

MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 

Test  &  Evaluation 

MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 

Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 

Development  Eng.  and  BTL  Development  Cost  Models,  Technomics,  Aug  90 

Kanter’s  Factors,  Cost  Factors  and  ^timating  Relationships,  Electronic  Sys  Division,  Apr  90 

Aircraft  Airframe  CERs,  RAND,  1987  CTotal  Level) 

_ Assessing  Acquisition  Schedules  for  Tactical  Aircraft,  IDA  1989 _ 


Training 


AFCAA,  Below  the  Line  Cost  Factors,  1998 
MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 
Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 
SE/PM  Database,  TASC,  1997 
Standard  Cost  Factors  Handbook,  NCCA,  1992 

Cost  Factors  for  A/C  and  Missiles,  Aeronautical  Systems  Division,  May  87 

CER  Develoment  for  R&D  Data  and  SE/PM,  Applied  Research,  Mar  90 

Development  Eng.  and  BTL  Development  Cost  Models,  Technomics,  Aug  90 

Ranter’s  Factors,  Cost  Factors  and  Mmating  Relationships,  Electronic  Sys  Division,  Apr  90 

Aircraft  Airframe  CERs,  RAND,  87  (Total  Level) 

AFCAA,  Below  the  Line  Cost  Factors,  1998 
MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 
Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 
Standard  Cost  Factors  Handbook,  NCCA,  1992 
HAPCA  data,  NAVAIR,  1991 

Cost  Factors  for  A/C  and  Missiles,  Aeronautical  Systems  Division,  May  87 

Development  Eng.  And  BTL  Development  Cost  Models,  Technomics,  Aug  90 

CER  Develoment  for  R&D  Data  and  SE/PM,  Applied  Research,  Mar  90 

Ranter’s  Factors  Cost  Factors  and  Estimating  Relationships,  Bectronic  Sys  Division,  Apr  90 

Aircraft  Airframe  CERs,  RAND,  87  (Total  Level) 

AFCAA,  Below  the  Line  Cost  Factors,  1998 
MACDAR  Fighter  Aircraft  Database,  Tecolote,  1 997 
Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 
Standard  Cost  Factors  Handbook,  NCCA,  1992 
HAPCA  data,  NAVAIR,  1991 

Cost  Factors  for  A/C  and  Missiles,  Aeronautical  Systems  Division,  May  87 

Development  Eng.  and  BTL  Development  Cost  Models,  Technomics,  Aug  90 

Ranter’s  Factors,  Cost  Factors  and  ^timating  Relationships,  Electronic  Sys  Division,  Apr  90 


Support  Equipment 


AFCAA,  Below  the  Line  Cost  Factors,  1998 
MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 
Advanced  Fighter  Aircraft  Cost  Model,  AFCAA,  1997 
Standard  Cost  Factors  Handbook,  NCCA,  1992 

Cost  Factors  for  A/C  and  Missiles,  Aeronautical  Systems  Division,  May  87 
Development  Eng.  And  BTL  Development  Cost  Models,  Technomics,  Aug  90 
CER  Develoment  for  R&D  Tooling  ,  Applied  Research,  Mar  90 

Ranter’s  Factors,  Cost  Factors  and  Estimating  Relationships,  Electronic  Sys  Division,  Apr  90 
VAMOSC 


MACDAR  Fighter  Aircraft  Database,  Tecolote,  1997 

Ranter’s  Factors,  Cost  Factors  and  Estimating  Relationships,  Electronic  Sys  Division,  Apr  90 
OP-20,  Obligated  Spend  Profiles,  NAVAIR,  annual 


AFI 65-503,  USAF  Cost  Planning  Factors 

VAMOSC 

Abides 

Tri-Service  LCC  Model,  EER  Systems,  Unknown 

LCC  Models  Reference  GuideASD,  Apr  83 

Modeling  the  Cost  of  Ownership  for  Aircraft,  RAND,  Aug  81 

Estimating  Recoverable  Spares  Investment,  RAND,  Aug  80 

DCA  Circular  600-60-1,  Cost  &  Planning  Factors,  TASC,  Mar  83 

Naval  Fixed  Wing  Aircraft  O&S  Cost  Estimating  Model,  Delta  Research,  1990 

Naval  Aircraft  Modification  Database,  MCR,  1996 

Line  Shut-Down  Study,  MCR,  1996 

C3  Platform  Integration  Cost  Model,  MCR,  1997 

Estimating  Annual  O&S  Cost,  Watson  Noah,  Jan  75 

Avionics  Parametric  Cost  Model,  ASD,  Feb  73 

PPR  Data  /  SDLMs  (Depot  Level  Maintenance),  NADOC,  annual 

OP-20,  Obligated  Spend  Profiles,  NAVAIR,  annual 
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IV.  ROTARY-WING  AIRCRAFT 
Theresa  O’Brien,  Air  Force  Cost  Analysis  Agency 


Aircraft  - 

Rotary  Wing 

PDRR 

EMD 

Production 

o&s 

Airframe 

11  1 

1  1 

Propulsion 

LI  1 

■■I 

■M 

1 

Subsystems  Giydraulics,  flight  controls...)  1  1  1  I  I 

L_L_ 

1 

Avionics  ^  1  1  1 

LT- 

1  . 

Software  IHH  IHZZi 

rr- 

Armament 

III 

11 

■M 

r~r~' 

Test  and  Evaluation 

. . 1_LJ 

p-  I  ' 

r-f- 

1.  1 

SE/PM 

L_H 

um 

L_H 

LI 

Data 

GIM 

.  1 

r 

Training 

HIM 

nn 

i  M  11  1 

Support  Equipment 

l_LJ 

1  1. 

J  1  1  1 

Spares 

L_LJ 

I L 

U_l  Li_l 

Survey  respondents  for  rotary-wing  aircraft  primarily  were  from  CEAC, 
AMCOM,  and  at  a  top  level  from  NAVAIR. 

The  Army  has  created  a  database  of  CCDRs  for  all  rotary-wing  aircraft. 
This  database  provides  valuable  historical  data  for  analogies  or  factors. 

Like  the  fixed-wing  aircraft,  composites  are  a  challenge  for  estimating 
the  Airframe  portion. 

The  Propulsion  area  has  a  current,  detailed  database  available  to  analyst. 
Again,  the  challenge  is  capturing  the  cost  to  enhance  existing  systems. 

The  areas  of  Subsystems  and  Avionics  are  not  coded  as  red  as  they  are 
for  fixed-wing  aircraft,  primarily  due  to  the  models  used  and  the  respondent’s 
comfort  level  (see  the  following  list  of  models  and  studies).  The  trouble  with 
the  existing  models  is  that  they  require  detailed  inputs  that  the  analyst  is 
typically  lacking  at  the  time  of  the  estimate. 

SE/PM,  Data,  Training,  and  Support  Equipment  are  coded  similarly 
based  on  the  databases  containing  the  historical  data  and  not  capturing 
acquisition  streamlining  initiatives  the  contractors  are  implementing. 

Finally,  a  comment  on  Support  Equipment.  One  remark  on  the  survey 
noted  the  vast  amount  of  equipment  required  to  support  rotary- wing  aircraft  and 
the  challenges  it  presents  to  the  estimator. 

I  would  now  like  to  turn  the  presentation  over  to  Rick  Collins  from 

NCCA. 


Airframe 

ACDB  Aircraft-Rotary  Wing,  SAIC,  1997 
Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 
Con^sitcs^otic  Materials  Database,  Tecolotc,  1997 
Advanced  Airframe  Structural  Materials,  RAND  Study,  1991 
ACDB  Aircraft-Rotary  Wing,  SAIC,  1997 

Cost  Considerations  for  LO  Technology  for  the  Comanche  Helo,  SAIC,  1994 

Dev.  of  Cost  Est.  Methodologies  for  Composite  Aircraft  Structures  and  Components,  LSA,  1988 

CERs  by  WBS  for  Selected  Helicopter  Systems,  CALIBER  Systems,  1991 

Propulsion 

ACDB  Aircraft-Rotary  Wing,  SAIC,  1997 

Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 

Aircraft  Gas  Turbine  engine  Acquisition  Costs,  Ketron,  1997 

CERs  by  WBS  for  Selected  Helicopter  Systems,  CALIBER  Systems,  1991 

Subsystems 

ACDB  Aircraft-Rotary  Wing,  SAIC,  1997 
Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 

Avionics 

ACDB  Aircraft-Rotary  Wing,  SAIC,  1997 
Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 

Parametric  Approach  to  Est.  Cost  of  Dev.  Eng.  ARI/87  TM-387,  Applied  Research  Inc.,  1987 
Electronics  Cost  Model  (TR-9505-01)  Technomics,  1996 

Parametric  Avionics/Electronics  Procurement  &  A/C  Retrofit  Cost  Study/  Vol  II,  General  Dynamics,  1984 
CERs  by  WBS  for  Selected  Helicopter  Systems,  CALIBER  Systems,  1991 
Organizational  Options  for  Common  Elec  Mgmt.,  IDA,  1992 
Tactical  Strategic  IR  Sensor,  Technomics,  1992 

Software 

Price  S,  Parametric  Review  of  Info  for  Costing  and  Evaluation  Software  Sizing  Model,  GE,  1997 
SEER  SEM,  Systems  Evaluation  and  Estimation  Resources-S/W  Est  Model,  Galorath,  1997 
Revic,  Software  Cost  Estimating  Model,  AFCAA,  Feb  91 

SASET,  Software  Architecture  Sizing  and  Estimating  Tool,  Martin  Marietta,  Apr  93 
_ Development  Support  Cost  Model  (TR9S0S-04),  Technomics,  1996 _ 


Armament 

ACDB  Aircraft-Rotary  Wing,  SAIC,  1997 
Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 
Test  &  Evaluation 

ACDB  Aircraft- Rotary  Wing,  SAIC,  1997 
Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 

SE/PM 

ACDB  Aircraft-Rotary  Wing,  SAIC.  1997 
Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 

Data 

ACDB  Aircraft-Rotary  Wing.  SAIC.  1997 
Rotary  Wing  Cost  Factor  Study,  SAIC,  1996 
HAPCA  data.  NAVAIR,  1991 

Training 

HAPCA  data.  NAVAIR,  1991 
Support  Equipment 

VAMOSC-AMSR 

OSMIS 

Force  Cost  Model,  CEAC,  1997 

Spares 

OP-20,  Obligated  Spend  Profiles,  NAVAIR,  annual 
CASA  Cost  Analysis  Strategy  Assement,  DSMC,  1997 

O&S 

VAMOSC/ OSMIS 

H^R  Data  /  SDLMs  (Depot  Level  Maintenance),  NADOC,  annual 
OP-20,  Obligated  Spend  Profiles,  NAVAIR,  annual 
Tri-Service  LCC  Model,  EER  Systems,  Unknown 
Modeling  the  Cost  of  Ownership  for  Aircraft,  RAND,  Aug  81 
Estimating  Annual  O&S  Cost,  Watson  Noah,  Jan  75 

Naval  Rotary  Wing  Aircraft  O&S  Cost  Estimating  Model,  Delta  Research,  1990 
_ force  Cost  Model,  CEAC.  1997 
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V.  ELECTRONICS 

Richard  Collins,  Naval  Center  for  Cost  Analysis 
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This  slide  depicts  the  assessment  of  electronics  cost  estimating 
capability.  The  assessment  is  based  on  inputs  from  representatives  of  nine  DoD 
and  two  private  sector  organizations,  including  the  following: 

•  AFCAA 

•  ESC/FMC 

•  CECOM 

•  NAVAIR 

•  NAVSEA 

•  NSWC/Dahlgren  Division 

•  NCCA 

•  OSD  CAIG 

•  BMDO 

•  Technomics,  Inc. 

•  Tecolote  Research,  Inc. 
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In  general,  the  assessment  tends  to  mirror  the  expectations  discussed  earlier. 
With  a  few  exceptions,  PDRR  is  rated  red-yellow,  EMD  is  rated  yellow, 
production  is  rated  yellow-green,  and  O&S  is  rated  yellow.  I  will  focus  my 
comments  on  the  significant  (with  respect  to  percent  of  total  cost)  areas  where  the 
assessment  differs  from  the  expectations,  but  I  want  to  make  some  general 
comments  before  doing  so.  First,  several  of  the  contributors  to  this  assessment 
warned  that  the  pace  of  electronics  technology  evolution  translates  to  cost  models 
having  short  shelf  lives  (i.e.,  two  to  five  years  depending  on  the  component). 
Second,  and  directly  related  to  the  first  comment,  it  is  imperative  that  electronics 
cost  estimating  methodology  incorporate,  to  the  extent  possible,  the  impact  of 
technology  trends. 

Next,  I  will  address  the  significant  areas  where  the  assessment  was  worse 
than  expected. 

•  Software:  A  number  of  factors  contributed  to  the  nearly  100-percent  red 
rating.  First,  with  respect  to  data,  the  quantity  and  quality  of 
development  and  maintenance  data  are  viewed  as  problematic.  Second, 
with  respect  to  methodology,  the  heavy  reliance  of  existing  models 
(public  domain  and  commercial)  on  subjective  inputs  is  viewed  as 
problematic.  Third,  with  respect  to  technical  definition,  the  uncertainty 
in  sizing  estimates  is  viewed  as  problematic. 

•  Platform  Integration  and  Installation:  The  rationale  for  the  assessment 
is  quite  simple — lack  of  understanding  of  the  explanatory  variables,  no 
compilation  of  data  and  no  methodology.  With  respect  to  the  data  void, 
cost  reports  typically  do  not  provide  the  visibility  required  to  isolate 
these  costs. 

Finally,  I  will  address  the  significant  areas  where  the  assessment  was  better 
than  expected. 

•  Processor  and  Display  and  Control  Hardware:  The  relatively  favorable 
assessment  is  directly  related  to  the  increasing  application  of 
commercial  off-the-shelf  (COTS)  equipment  for  these  functions. 
Specifically,  with  respect  to  recurring  hardware  costs,  availability  of 
COTS  price  information  and  knowledge  of  COTS  price  trends  for  these 
types  of  equipment  is  the  basis  for  the  positive  perspective. 

The  next  slides  are  a  representative  (but  not  exhaustive)  compilations  of  the 
electronics  estimating  databases  and  methodology  that  the  eleven  contributing 
organizations  use. 
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Electronics  Estimating  Databases  &  Methodology 


Hardware 

Acquis  The  Black  Box  Estimators  Electronics  Cost  Models,  Tecolote,  Nov  87 
Communications  and  Electronics  Cost  Model,  Technomics,  Oct  96 
Electronics  Cost  Model,  Technomics,  Jan  96 
CERs  for  Shipboard  RF  Sensors,  Technomics,  Jan  92 
Radar  Engineering  and  Cost  Tool  (REACT),  Tecolote,  Dec  93 
COTS  Electronics  Cost  and  Technical  Database,  LSA  Inc.,  May  96 
Digital  Signal  and  Data  ftocessor  Model,  Tecolote,  Sep  91 
Ground  Based  Radar  Signal  and  Data  Processor  CERs,  Tecolote,  Dec  93 
Radar  Receiver  Estimating  Model,  Tecolote,  Feb  96 
T/R  Module  Cost  Model,  Tecolote,  1996  (update  due  Mar  98) 

Microwave  and  Digital  Cost  Analysis  Model  (MADCAM),  Tecolote,  Mar  96  (update  due  Apr  98) 

Circuit  Card  Assembly  Database,  NSWC/Crane  Division,  Oct  96 

Estimating  Nonrecurring  Design  Cost  and  Development  Time  in  Avionics  ftograms,  MCR,  Dec  97 
Development  Engineering  and  Below-the-Line  Development  Cost  Models,  Technomics,  Aug  90 
Analyses  of  the  Relationship  Between  Development  and  Production  Unit  Hardware  Costs,  NCCA,  Jan  94 
An  Analysis  of  Electronics  Systems  Learning  and  Rate  Effects,  NCCA,  May  93 
Procedures  for  Estimating  Life  Cycle  Costs  of  Electronic  Combat  Equipment,  Rand,  Feb  88 
PRICE  H 

O&S  Visibility  and  Management  of  Operating  and  Support  Costs  (VAMOSC)  Database 

Operating  and  Support  Cost  Analysis  Model  (OSCAM)-Ship  Systems,  NCCA  &  HVR  Consulting,  Jan  98 

COTS  Electronics  Technology  Assessment/Refresh  Cost  Model,  NSWC/Crane  Division,  Sep  97 

Cost  of  Manpower  Estimating  Tool  (COMET),  NCCA  &  SAG,  Dec  97 

Army  Military-Civilian  Cost  System 

C3  Hardware  Maintenance  Statistical  Analysis,  MCR,  97 

PRICE  HL 


Electronics  Estimating  Databases  &  Methodology  (2) 

Software 

Acquis  Software  Development  Cost  Estimating  Handbook,  NCCA,  Jan  98 
SMC  Software  Database,  MCR  Federal,  annual  updates 
CERs  for  Software  Development,  Aerospace  Corp.,  May  96 
PRICE  S,  SEER-SEM  &  Riefer  models 

CERs  for  Space-Based  Systems,  Defense  Communications  Agency-DC,  Apr  91 
ESC/FMC  Software  Factors  and  Estimating  Relationships,  Dec  95 

Platform  Integration  Sc  ftistallation 

C3  Platform  Integration  Model,  MCR,  97 
Electronic  Subsystem  Integration  Estimator,  TASC,  Jul  85 
System  Engineering/Integration  CER  Update,  Tecolote,  Aug  92 
PRICE  H 

SE/PM,  ST&E  and  Other  Support  Elements 

Acquis  ESC/FMC  Acquisition  Support  CERs  and  Factors,  Tecolote,  Sep  93 
System  Engineering/Integration  CER  Update,  Tecolote,  Aug  92 

Development  Engineering  and  Below-the-Line  Development  Cost  Models,  Technomics,  Aug  90 
SESAME  Model 

Electronic  Equipment:  Cost  of  Spares  Research,  Cygnus,  95 
O&S  Visibility  and  Management  of  Operating  and  Support  Costs  (VAMOSC)  Database 

Operating  and  Support  Cost  Analysis  Model  (OSCAM)-Ship  Systems,  NCCA  &  HVR  Consulting,  Jan  98 
Cost  of  Manpower  Estimating  Tool  (COMET),  NCCA  &  SAG,  Dec  97 
Army  Military-Civilian  Cost  System 
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VI.  SfflPS 

Richard  Collins,  Naval  Center  for  Cost  Analysis 
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This  slide  depicts  the  assessment  of  ships  cost  estimating  capability.  The 
assessment  is  based  on  inputs  from  representatives  of  four  DoD  organizations, 
including  the  following: 

•  NAVSEA 

•  NSWC/Carderock  Division 

•  NCCA 

•  OSDCAIG 

The  assessment  for  PDRR  and  EMD  tends  to  mirror  the  expectations 
discussed  earlier.  With  a  few  exceptions,  PDRR  is  rated  red-yellow  and  EMD  is 
rated  yellow.  The  assessment  for  production  tends  to  differ  from  the  expected 
yellow-green;  it  is  principally  a  mix  of  either  yellow  or  green.  Finally,  the 
assessment  for  O&S  differs  from  the  expected  yellow  in  several  areas  and  is 
red- yellow.  I  will  focus  my  comments  on  the  significant  (with  respect  to  percent 
of  total  cost)  areas  where  the  assessment  differs  from  the  expectations. 


Next,  I  will  address  the  significant  areas  where  the  assessment  was  worse  than 
expected. 

•  Software:  A  number  of  factors  contributed  to  the  nearly  100-percent  red 
rating.  First,  with  respect  to  data,  the  quantity  and  quality  of  development 
and  maintenance  data  are  viewed  as  problematic.  Second,  with  respect  to 
methodology,  the  heavy  reliance  of  existing  models  (public  domain  and 
commercial)  on  subjective  inputs  is  viewed  as  problematic.  Third,  with 
respect  to  technical  definition,  the  uncertainty  in  sizing  estimates  is  viewed 
as  problematic. 

•  Integration/Engineering  Hardware.  Ship  Assembly  and  Support  Services 
Hardware.  SE/PM.  System  Test  and  Evaluation.  Training,  and  Data:  The 
rationale  for  the  assessment  is  quite  simple — lack  of  understanding  of  the 
explanatory  variables  resulting  in  little  to  no  meaningful  methodology. 

Finally,  I  will  address  the  significant  areas  where  the  assessment  was  better  than 
expected. 

•  Hardware  and  Spares  and  Repair  Parts:  Hull,  propulsion,  electric,  auxiliary, 
and  outfittings  and  furnishings  are  viewed  as  less  complex  subsystems  that 
are  better  understood  than  the  more  complex  electronics-oriented  command 
and  surveillance  and  armament  subsystems.  This  same  rationale  applies  to 
the  spares  and  repair  parts  associated  with  these  subsystems. 
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Ships  Estimating  Databases  &  Methodology 


Hardware 

Acquis  Historical  Ship  Cost  (HCOST)  Database,  NAVSEA 
Unit  Price  Analysis  (UPA)  Model,  NAVSEA 

Product  Oriented  Design  &  Construction  (POD AC)  Cost  Model,  NSWC/Carderock  Division  et  al.,  Feb  97 
ASSET/ACE  Interface(AACEI)  Cost  Model,  Tecolote,  Sep  96 

Nuclear  Attack  Submarine  Performance-Based  Cost  Model,  NSWC/Carderock  Division,  Jun  96 
U.S.  Naval  Vessels  Detail  Design  Cost  Estimating  Models,  Gibbs  &.  Cox,  Jun  91  &  Jul  92 
U.S.  Naval  Surface  Ships  Construction  Cost  Model,  Gibbs  &  Cox,  Jun  94 
U.S.  Naval  Ship  Upgrade  Construction  Cost  Model,  Gibbs  &  Cox,  Aug  95 
Ship  Construction  Cost  &  Technical  Database,  NCCA,  Feb  93 
Ship  Construction  Manhour  &  Material  Cost  Learning  Curves,  NCCA,  Jan  94 
O&S  Visibility  and  Management  of  Operating  and  Support  Costs  (VAMOSC)  Database 

Operating  and  Support  Cost  Analysis  Model  (OSCAM)-Ships,  NCCA  &  HVR  Consulting,  Jan  98 
Operating  and  Support  Cost  Analysis  Model  (OSCAM)-Ship  Systems,  NCCA  &  HVR  Consulting,  Jan  98 
Cost  of  Manpower  Estimating  Tool  (COMET),  NCCA  &  SAG,  Dec  97 

Software 

Acquis  Software  Development  Cost  Estimating  Handbook,  NCCA,  Jan  98 
SMC  Software  Database,  MCR  Federal,  annual  updates 
CERs  for  Software  Development,  Aerospace  Corp.,  May  96 
PRICE  S,  SEER-SEM  &  Riefer  models 

SE/PM»  ST&E  and  Other  Support  Elements 

Acquis  Historical  Ship  Cost  (HCOST)  Database,  NAVSEA 
Unit  Price  Analysis  (UPA)  Model,  NAVSEA 

O&S  Visibility  and  Management  of  Operating  and  Support  Costs  (VAMOSC)  Database 

Operating  and  Support  Cost  Analysis  Model  (OSCAM)- Ships,  NCCA  &  HVR  Consulting,  Jan  98 
Operating  and  Support  Cost  Analysis  Model  (OSCAM)-Ship  Systems,  NCCA  &  HVR  Consulting,  Jan  98 
Cost  of  Manpower  Estimating  Tool  (COMET),  NCCA  &  SAG,  Dec  97 

This  slide  is  a  representative  (but  not  exhaustive)  compilation  of  the 
ships  estimating  databases  and  methodology  that  the  four  contributing 
organizations  use. 
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VII.  MISSILES 

Richard  Bishop,  US  Army  Cost  and  Economic  Analysis  Center 
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Four  organizations  sent  us  data:  (1)  Army  Aviation  and  Missile 
Command;  (2)  Army  Space  and  Missile  Defense  Command;  (3)  NAVAIR;  and 
(4)  AFCAA. 

Overall,  the  missiles  area  is  in  relatively  good  condition  (yellow-green). 
This  can  be  attributed  to  the  fact  that  CCDR  data  have  been  required  and 
collected  by  the  Army,  Navy,  and  Air  Force.  In  addition,  many  missile  studies 
and  models  have  been  developed. 

Production  has  a  great  deal  of  green. 

Launcher  Equipment  is  red  in  PDRR  and  EMD. 

Surprises  are  the  red  in  Propulsion  and  Airframe  and  no  red  in  Guidance 
and  Control. 

Propulsion  methodology  is  aging  and  needs  to  be  updated. 

Airframe  is  red,  probably  because  of  Composite  Materials. 

Guidance  and  Control  Seekers  was  a  big  unknown  area  a  few  years  ago, 
but  we  have  studied  them  and  now  have  actuals  for  imaging  infra-red.  Also, 
millimeter  wave  seekers  are  not  the  big  mystery  that  they  once  were. 
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VIII.  SURFACE  VEfflCLE  SYSTEMS 
Richard  Bishop,  US  Army  Cost  And  Economic  Analysis  Center 
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Data  are  from  the  Army  Tank  and  Automotive  Command  and  CEAC. 

The  overall  comment  from  TACOM  was  that  they  have  data  and 
methods  for  all  areas  except  for  materials  that  push  the  state  of  the  art. 

Auxiliary  Automotive  is  red  in  all  phases  except  O&S;  we  have  no  data 
for  these  sub-systems. 

Automatic  Loader:  we  have  no  data  for  the  few  previous  Army  systems. 

For  Integration  and  Assembly,  we  lack  the  appropriate  level  of  detail 
and  have  no  confidence  in  parametric  methods. 

The  OSMIS  database  has  historical  data  for  most  of  the  other  items.  The 
CEAC  OSMIS  team  members  believe  they  can  develop  a  better  estimate  at 
PDRR  abd  EMD  than  most  can  at  the  production  milestone. 
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SURFACE  VEfflCLES  (p.2) 
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Initial  Spares  &  Repair  Parts 

Training  is  red  at  PDRR  because  training  devices  are  ill  defined  at  this 
point  in  time. 

Initial  Spares  and  Repair  Parts:  red  until  O&S. 

There  is  a  lack  of  data  at  the  level  required.  The  SESAME  model 
requires  extensive  data  input  and  the  proposed  simplified  SESAME  Model  still 
needs  seventeen  detailed,  accurate  inputs  for  a  parametric  estimate  in  which 
there  is  little  confidence  until  actuals  are  available.  The  OSMIS  Team 
recommends  using  analogous  OSMIS  data  for  projection. 

Peculiar  and  Common  Support  Equipment  categories  are  red  for  O&S 
due  to  a  lack  of  sufficient  detail  in  the  historical  data. 
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IX.  OSD  PERSPECTIVE 

Vance  Gordon,  Office  of  the  Deputy  Director  (Resource  Analysis), 
Program  Analysis  and  Evaluation 
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This  slide  summarizes  my  colleagues’  presentations.  As  Steve  Balut 
predicted,  the  uncertainty  of  our  estimates  is  greatest  at  Milestones  I  and  H,  and 
decreases  as  we  approach  production.  This  is  not  new  news,  but  it  is  a  more 
systematic  view  than  we  have  previously  been  able  to  present. 

It  is,  moreover,  important  to  bear  in  mind  that  this  picture  captures  our 
uncertainty  at  each  milestone.  If  a  similar  chart  were  prepared  for  our 
uncertainty  at  Milestone  I  of  the  costs  of  each  phase,  it  would  be  far  more  red 
than  this  one.  As  the  next  slide  shows,  this  should  give  us  strong  reason  for 
concern. 
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The  challenges  we  can  expect  to  face  in  the  next  decade  are  in  precisely 
that  area  in  which  our  current  tools  are  weakest;  the  estimation  of  life-cycle 
costs  for  programs  in  the  early  stages  of  development. 

This  slide  depicts  the  distribution  of  MDAPs  at  three  times:  in  the 
middle  of  the  Reagan  defense  build-up,  at  the  end  of  the  Cold  War,  and  today. 
In  contrast  to  the  earlier  years,  we  have  today  a  preponderance  of  programs  in 
the  early  stages  of  development.  In  fact,  the  distribution  we  see  today  has  only 
one  historical  precedent,  in  the  set  of  programs  on  hand  in  the  late  1970s,  which 
came  to  adulthood  in  the  distribution  we  see  in  1985. 

As  shown  in  the  next  slide,  today’s  distribution  of  programs  poses  a 
future  funding  problem  as  well  as  a  cost-estimating  challenge. 
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Projected  Procurement  Costs  for  Current  MDAPS 
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This  slide  shows,  in  FY  1997  dollars,  the  currently  projected 
procurement  costs  for  the  MDAPs  shown  in  the  last  slide.  The  total  annual 
requirement  during  FY  2006-2012  is  about  $13  billion  more  than  we  will  spend 
on  MDAP  procurement  this  year,  and  leaves  no  room  for  error  within  the 
targeted  increase  in  our  procurement  funding,  which  is  shown  by  the  dotted 
line. 

It  is  therefore  likely  that  hard  decisions  will  be  required  to  fit  the  future 
defense  acquisition  program  within  the  available  funds.  The  wisdom  of  those 
decisions  will  depend  absolutely  on  the  realism  of  our  cost  estimates,  which  in 
turn  depends  on  the  quality  of  the  tools  we  are  able  to  bring  to  bear. 

It  is  for  that  reason  that  we  have  embarked  on  the  current  effort  to 
define  and  improve  the  status  of  DoD  cost  research. 
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X.  CLOSING 

Stephen  J.  Balut,  Institute  for  Defense  Analyses 


WHATS  NEXT 

m  Document  this  assessment 
m  Update  Cost  Research  Roadmap 
m  Review  ongoing  cost  research 

-  Catalog  projects 

B  Prepare  FY  Cost  Research 
Program 

Decentralized 

-  Informed 


We’re  going  to  open  the  floor  to  questions  in  a  moment.  First,  I  want  to 
let  you  know  what  comes  next  for  cost  research  in  the  DoD,  and  where  you  can 
get  more  information  about  cost  research. 

Our  panel  will  document  the  assessment  you’ve  just  seen  and  place  it  on 
the  internet.  Documentation  will  include  the  slides  you  saw  here,  and  also 
backup  materials  used  to  develop  the  scores. 

Over  the  next  few  months,  we  will  be  updating  the  DoD  Cost  Research 
Plan  in  light  of  what  you’ve  seen  here  today.  The  updated  Plan  is  intended  to 
guide  subsequent  research  investments  to  areas  of  greatest  need. 

Next,  we  will  review  ongoing  research  activities  at  the  IDA/CAIG  Cost 
Research  Symposium  to  be  held  in  May.  A  draft  catalog  of  projects  in  progress 
or  planned  will  be  given  to  participants  at  that  time.  This  catalog  will  be 
finalized  in  August  and  will  be  placed  on  the  web. 

Then  we  get  to  the  real  purpose  of  this  cycle  of  annual  planning.  During 
the  summer,  sponsors  will  select  topics  for  study  during  FY  1999.  They  will 
make  these  selections  in  a  decentralized  way,  but  their  decisions  will  be 
informed — informed  by  the  assessments  at  DoDCAS,  the  updated  roadmap,  and 
knowledge  of  the  current  status  of  ongoing  cost  research  as  contained  in  the 
catalog. 
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COST  RESEARCH  INFO 

■  Research  results 

-  DTIC 

-  WWW.ASAFM.ARMY.MIL/CEAC# 

-  WWW.NCCA.NAVY, MIL 

-  WWW.AFCAA.AF.MIL 

-  WWW.RA.PAE.OSD.MIL/ADODCAS 

■  Ongoing  research 

-  IDA  Catalog  on  Web 

-  Cost  Research  Database  under  development 
1  Documentation  of  this  assessment 

-  Will  be  distributed  and  put  on  web 

■  Update  to  S-Year  Cost  Research  Roadmap 

-  Will  be  distributed  and  put  on  web 


This  slide  shows  where  you  can  go  to  get  more  information  on  cost 
research.  Many  completed  studies  are  sent  to  DTIC.  Others  are  not  and  are 
sometimes  made  available  by  the  sponsoring  office  directly.  In  some  cases, 
results  are  placed  on  web  sites.  This  slide  lists  some  of  these  sites. 

The  only  place  to  get  a  broad  view  of  ongoing  research  is  in  the  catalog 
produced  in  conjunction  with  the  Cost  Research  Symposium.  The  catalog  is 
placed  on  the  web.  For  example,  the  1997  catalog  is  now  on  the  OSD 
ADODCAS  site  in  Adobe  Acrobat  format. 

Also,  the  CAIG  is  developing  a  cost  research  database  and  will  make  it 
available  to  users  when  completed. 

Documentation  of  the  assessment  you  heard  today  will  be  placed  on  the 
web — on  the  ADODCAS  site.  The  update  to  the  DoD  Six-Year  Cost  Research 
Plan  will  also  be  put  on  that  web  site. 
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APPENDIX 


OFFICES  CONTRIBUTING  TO  THE  ASSESSMENTS 


Space  Systems 

Air  Force  Cost  Analysis  Agency  (AFCAA) 

Air  Force  Space  and  Missile  Systems  Center  (AFSMC) 

Fixed  Wing  Aircraft 

Air  Force  Cost  Analysis  Agency  (AFCAA) 

Naval  Air  Systems  Command  (NAVAIR) 

Naval  Center  for  Cost  Analysis  (NCCA) 

Air  Force  Materiel  Command/Aeronautical  Systems  Center  (HSC/EMP) 
Rotary  Wing  Aircraft 

Army  Cost  &  Economic  Analysis  Center  (CEAC) 

Army  Aviation  and  Missile  Command  (AAMC) 

Naval  Air  Systems  Command  (NAVAIR) 

Electronics 

Air  Force  Cost  Analysis  Agency  (AFCAA) 

Air  Force  Electronics  Systems  Center  (ESC/FMC) 

Army  Coimnunications  &  Electronics  Command  (CECOM) 

Naval  Air  Systems  Command  (NAVAIR) 

Naval  Sea  Systems  Command  (NAVSEA) 

Naval  Surface  Warfare  Center  Dahlgren  Division  (NSWCDD) 

Naval  Center  for  Cost  Analysis  (NCCA) 

OSD  Cost  Analysis  Improvement  Group  (CAIG) 
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Ballistic  Missile  Defense  Organization  (BMDO) 

Technomics,  Inc. 

Tecolote  Research,  Inc. 

Ships 

Naval  Sea  Systems  Command  (NAVSEA) 

Naval  Surface  Warfare  Center  Carderock  Division  (NSWCCD) 
Naval  Center  for  Cost  Analysis  (NCCA) 

OSD  Cost  Analysis  Improvement  Group  (CAIG) 

Missiles 

Air  Force  Cost  Analysis  Agency  (AFCAA) 

Army  Aviation  &  Missile  Command 

Army  Cost  &  Economic  Analysis  Center  (CEAC) 

Army  Space  and  Missile  Defense  Command 
Naval  Air  Systems  Command  (NAVAIR) 

Navy  Center  for  Cost  Analysis  (NCCA) 

Surface  Vehicle  Systems 

Army  Tank  -  Armament  and  Automotive  Command  (ATAAC) 
Army  Cost  &  Economic  Analysis  Center  (CEAC) 

OSD  Perspective 

Office  Secretary  of  Defense  (OSD  (PA&E)) 
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ABBREVIATIONS 


AFCAA 

Air  Force  Cost  Analysis  Agency 

ASC 

Aeronautical  Systems  Center 

BMDO 

Ballistic  Missile  Defense  Organization 

C/SSR 

Cost/Schedule  Status  Report 

C3 

command,  control,  and  communications 

CAIG 

Cost  Analysis  Improvement  Group 

CATV 

cost  as  an  independent  variable 

CCDR 

Contractor  Cost  Data  Reporting 

CEAC 

Cost  and  Economic  Analysis  Center 

CECOM 

Army  Communications  and  Electronics  Command 

CER 

cost-estimating  relationship 

COTS 

commercial  off-the-shelf 

CPR 

Cost  Performance  Report 

CR 

cost  research 

DAB 

Defense  Acquisition  Board 

DoD 

Department  of  Defense 

DoDCAS 

DoD  Cost  Analysis  Symposium 

DTIC 

Defense  Technical  Information  Center 

EELV 

Evolved  Expendable  Launch  Vehicle 

EMD 

Engineering  and  Manufacturing  Development 

ESC/FMC 

Air  Force  Electronics  Systems  Center 

FFRDC 

Federally  Funded  Research  and  Development  Center 

IDA 

Institute  for  Defense  Analyses 

JSF 

Joint  Strike  Fighter 

LRIP 

low-rate  initial  production 

MDAP 

Major  Defense  Acquisition  Program 

MS 

Milestone 

NAFCOM 

NAS  A/ Air  Force  Cost  Model 

NAVAIR 

Naval  Air  Systems  Command 

NAVSEA 

Naval  Sea  Systems  Command 

NCCA 

Naval  Center  for  Cost  Analysis 

NSWC 

Naval  Surface  Weapons  Center 
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o&s 

operating  and  support 

OSD 
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